Using ChatGPT

I've been mulling over different use case experiments I can undertake with the ChatGPT-3 language model, and last night - lying in bed - I had a light-bulb moment.

What about giving ChatGPT-3 a first crack at triaging bugs we raise as a normal part of the software development lifecycle?

Thanks to awesome integration between Azure DevOps and Power Automate, I was able to set up the below Power Automate flow which triggers any time a new bug is raised:

The key for me was prefacing the bug notes with a natural language intro to help make the response suitable for posting back to Azure DevOps:

And the results are fairly impressive! Here's a bug I raised, and a few seconds later I get an update from ChatGPT-3 with some triage notes:

My bug description (which actually happened to me today)

Deploying an AWS SAM template via an Azure devops build pipeline failed with the following exception: Unknown error executing command: Unable to cast object of type 'YamlDotNet.RepresentationModel.YamlScalarNode' to type 'YamlDotNet.RepresentationModel.YamlMappingNode'.

  at Amazon.Lambda.Tools.TemplateProcessor.YamlTemplateParser.UpdatableResources()+MoveNext() in C:\build\src\Amazon.Lambda.Tools\TemplateProcessor\YamlTemplateParser.cs:line 68

And my ChatGPT-3 analysis:

ChatGPT analysis: To investigate this error, I suggest examining the YAML template to determine which scalar nodes are being cast to YamlMappingNode. It might also be useful to look at the source code of the Amazon.Lambda.Tools.TemplateProcessor library to understand how it handles the YAML. Additionally, if the same error is generated when other YAML templates are deployed, it might be a good idea to check the version of the Amazon.Lambda.Tools.TemplateProcessor library that is being used.

It's not the worst advice! (It's also - possibly - no the best advice, but there's a lot of tweaking that could be done to improve this).

As always, feel free to leave comments and questions!